Method of processing images, notably from night vision systems and associated system

ABSTRACT

The general field of the invention is that of the methods of processing an initial digital image I IN (p) made up of pixels (p). The method according to the invention comprises the following successive steps:
         Step  1 : calculating the logarithm of the initial image in order to obtain a first intermediate image Log(I IN (p));   Step  2 : filtering said first intermediate image using a low-frequency filter in order to obtain a second intermediate image BF(p);   Step  3 : determining the value of the minimum intensity and the value of the maximum intensity in the low-frequency image;   Step  4 : calculating a third intermediate image Log(I OUT (p)) using the following linear combination:
 
Log( I   OUT ( p ))=[Log( I   IN ( p )− BF ( p )]·[ K 1· BF ( p )+ K 2]+ K 3· BF ( p )+ K 4
   K1, K2, K3 and K4 being constants;   Step  5 : calculating the final image I OUT (p) by applying the exponential function to the third intermediate image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention is that of the methods for improving imagesfrom real-time and high dynamic low-light level digital video imagingsystems. These imaging systems are notably used at night. There arenumerous uses, in particular in the aeronautical field.

2. Description of the Prior Art

This type of system provides images which are not necessarily ofextremely good quality, given the low luminance level of the scene.There are a certain number of image processing techniques for increasingthe quality of the image, in particular by increasing or enhancing thecontrast in the low grey levels.

The following contrast enhancement methods are noted:

-   -   enhancement based on global histogram rearrangement;    -   enhancement based on local histogram rearrangement;    -   enhancement based on frequency approaches;    -   enhancement based on illumination/reflectance approaches;    -   enhancement based on the “Retinex” human eye biological        function.

However, the images from the low-light level digital video imagingsystems are particularly tricky to process insofar as they are at highresolution, highly dynamic, extremely noisy and delivered at a highrate. Therefore, the conventional image processing techniques mentionedabove all have one or more of the following disadvantages:

-   -   degradation of the scenes having extremely wide luminous        dynamics by the saturation of the overexposed areas of the        scene;    -   large increase in the already present digital noise due to the        low-light level environment;    -   appearance of processing “artefacts” restricting the scene        comprehension;    -   reduction in the resolution of the initial image;    -   processing incompatibility with a high image rate real-time        operation;    -   no preservation of the overall appearance of the scene.

SUMMARY OF THE INVENTION

The image processing method according to the invention does not havethese disadvantages, it is a homomorphic filtering method. Indeed, theprocessed image takes into account the initial image and thelow-frequency components of this image such as to increase the contrastof the image without increasing the noise thereof, the final image,therefore, being more easily legible. Moreover this algorithm isparticularly suitable for the enhancement of contrast of an image withinthe context of image dynamics matching. Noted, for example, is thetransition from 10 bit, or more, coded image dynamics coming from thesensor to an 8 bit coded final image intended to be displayed on animaging device. Indeed, since the processing algorithm is based oncalculations using floating point or fixed point of chosen depth forreal-time appearances, it is then possible to profit from the entiredynamics offered by the sensor during processing, then to impose outputdynamics fixed to the algorithm during the last conversion. Therefore,this algorithm can, advantageously, be carried out within the context ofreplacing a conversion table matching dynamics between a sensing deviceand the display thereof.

More precisely, the subject matter of the invention is a method ofprocessing an initial digital image I_(IN)(p) made up of pixels (p),said method comprising at least the following successive steps:

Step 1: calculating the logarithm of the initial image in order toobtain a first intermediate image Log(I_(IN)(p));

Step 2: filtering said first intermediate image using a low-frequencyfilter in order to obtain a second intermediate image BF(p);

Step 3: determining the value of the minimum intensity (Min_(BF)) andthe value of the maximum intensity (Max_(BF)) in the low-frequency imageobtained in the previous step;

Step 4: calculating a third intermediate image Log(I_(OUT)(p)) using thefollowing linear combination:Log(I _(OUT)(p))=[Log(I _(IN)(p)−BF(p)]·[K1·BF(p)+K2]+K3·BF(p)+K4

K1, K2, K3 and K4 being constants dependent upon the difference betweenthe value of the maximum intensity (Max_(BF)) and the value of theminimum intensity (Min_(BF)) and the value of the minimum intensity(Min_(BF));

Step 5: calculating the final image I_(OUT)(p) by applying theexponential function to the third intermediate image Log(I_(OUT)(p)).

Advantageously, the filter used in step 2 is a Gaussian filter, thesigma of which is greater than 30.

Advantageously, with some processing architectures, step 2 includes thefollowing three sub-steps:

Step 2.1: pyramidal decomposition of the first intermediate image intofirst sub-sampled intermediate images Log(I_(INSSE)(p));

Step 2.2: filtering a first sub-sampled intermediate image using thelow-frequency filter in order to obtain a second sub-sampledintermediate image BF(p)_(SSE);

Step 2.3: pyramidal recomposition of the second sub-sampled intermediateimage in order to obtain the second intermediate image BF(p).

Advantageously, in the case where the method is used to process a seconddigital image immediately succeeding a first digital image, both imagesbelonging to a video sequence, at step 3 of the method of processing thesecond digital image, the value of the minimum intensity (Min_(BF)) andthe value of the maximum intensity (Max_(BF)) of the second digitalimage are chosen to be equal to the value of the minimum intensity(Min_(BF)) and the value of the maximum intensity (Max_(BF)) of thefirst digital image.

The invention also relates to a night vision system including alow-light level image capturing device and means for processing theimages from said device, characterized in that said means implement theprocessing method described above.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood and other advantages will emergeupon reading the following nonlimiting description with reference to theappended figures wherein:

FIG. 1 shows the various steps of the image processing method accordingto the invention;

FIG. 2 shows an alternative of the second step of the image processingmethod according to the invention.

DETAILED DESCRIPTION

A digital image is made up of an array of pixels p arranged in rows andin columns. Each row and each column can include several hundred toseveral thousand pixels. Each pixel is digitally coded. In the case of amonochrome image, the digital coding corresponds to a grey level or“GL”. In the following description, it is assumed that the image ismonochrome. The method is used, of course, for coloured images, acoloured image being the superposition of three monochrome images inthree different spectral bands.

The image of a scene I_(IN)(p) can be described as the product of anillumination or lighting component E(p) and a reflectance componentR(p). Therefore, an initial image I_(IN)(p) can be written as:I _(IN)(p)=R(p)·E(p)

The core of the method is to separately and independently process thesetwo components, then recombine them in order to obtain a better qualityprocessed image. The procedure is as follows. Firstly, a logarithmictransformation “Log” is applied to the image. Therefore, this gives:

Log(I_(IN)(p))=Log(R(p)·E(p))=Log(R(p))+Log(E(p)) according to thewell-known properties of the logarithms.

Generally, the reflectance component includes the high frequencies andthe illumination component includes the low frequencies. Therefore, byusing processing which selects the low frequencies, the illuminationcomponent Log(E(p)) can be isolated and specific processing can beapplied thereto which is intended, for example, to reduce the largeillumination variations resulting from a dramatic variation in the gainof a capturing system. The processed components Log(R(p)) and Log(E(p))are then recombined in order to obtain the following functionLog(I_(out)(p)), I_(out)(p) corresponding to the logarithmictransformation of the final processed image. It is, therefore,sufficient to apply the exponential function to this logarithmic imagein order to obtain the final processed image of “real” value, with afloating point or fixed point of sensibly chosen depth, which can bewritten as:Exp[Log(I _(out)(p))]=I _(out)(p)

Then, all that needs to be done is to use a suitable conversion table tosample this real final image into a chosen dynamics image. Within thecontext of a vision system, these are the dynamics of the display.

More precisely, FIG. 1 shows the various steps of the method forprocessing a pixellated digital image according to the invention. Theseare as follows:

Step 1 “LOG”: calculating the logarithm of the initial image I_(IN)(p)in order to obtain a first intermediate image Log(I_(IN)). The logarithmcalculated in this manner assigns to each pixel p “real” values whichare simulated by numbers with a floating point or fixed point of asensibly selected “depth”. The depth corresponds to a selected number ofbits.

Step 2 “FILTERING”: filtering said first intermediate imageLog(I_(IN)(p)) using a low-frequency filter in order to obtain a secondintermediate image BF(p). The second intermediate image corresponds moreor less to the illumination component of the image as seen above.

The filter used is preferably a Gaussian filter, the sigma of which isgreater than 30. More precisely, this sigma is strongly dependent uponthe angular resolution of the capturing system. In the context of thecapturing systems of the current night vision systems, a “good” sigma isapproximately 60. Filtering consists in convolving the firstintermediate image with a Gaussian function. It is known that one of theproperties of the Gaussian function is a Fourier transform which is alsoa Gaussian function. Therefore, this does not have any rebound in thefrequency domain unlike other functions such as the “door” function, forexample.

Step 3 “MIN/MAX”: determining the value of the minimum intensity(Min_(BF)) and the value of the maximum intensity (Max_(BF)) in thepixels of the low-frequency image BF(p) obtained in the previous step.

Step 4 “LINEAR.C”: calculating a third intermediate imageLog(I_(OUT)(p)) using the following linear combination:Log(I _(OUT)(p))=[Log(I _(IN)(p))−BF(p)]·[K1·BF(p)+K2]+K3·BF(p)+K4

K1, K2, K3 and K4 being first order dependent constants for:

-   -   the inverse of the difference between the value of the maximum        intensity and the value of the minimum intensity which is:        1/(Max_(BF)−Min_(BF));    -   the value of the maximum intensity Max_(BF) and;    -   the value of the minimum intensity Min_(BF)

For example, the constants K1, K2, K3 and K4 are:

-   -   K1=(1−α)/(Max_(BF)−Min_(BF))    -   K2=α−K1·Min_(BF)    -   K3=(Max_(BF)−β)/(Max_(BF)−Min_(BF))    -   K4=β−K3·Min_(BF)    -   α and β being adjustable constants.

This linear combination allows the reflectance component which is[Log(I_(IN)(p))−BF(p)] and the illumination component which is BF(p) tobe weighted separately, and account to be taken of the lightingamplitude which is (Max_(BF)−Min_(BF)) and of the lighting levelscorresponding to the values of the maximum intensity Max_(BF) and of theminimum intensity Min_(BF).

Step 5 “EXP”: calculating the final image I_(OUT)(p) by applying theexponential function to the third intermediate image Log(I_(OUT)(p)).This function is tabulated, for each pixel, to the “bits per pixel” or“BPP” precision required at processing output.

If the processed digital images are video images requiring real-timeprocessing, it is imperative that the duration of the processing of eachimage is at the rate of acquisition, i.e. a few dozen milliseconds. Inthis case, it is important to optimize the calculating time and thevarious operations for processing the image. In this case, step 2 caninclude an image pyramidal decomposition sub-step. The pyramidaldecomposition takes place in successive steps. At each step, adjacentpixels are grouped together in blocks including numbers of pixels thatare increasingly greater according to grouping-together laws taking intoaccount the intensity variations of each pixel. Therefore, the higher upthe “pyramid”, the fewer blocks it contains and the more the blockscomprise a large number of pixels. The pyramidal decomposition can occurwith loss of information or without loss of information.

To find the initial image, each block is decomposed, in successivesteps, until the original pixel number is found. It is understood that,if processing is carried out on the high rank blocks, then a largenumber of pixels can be processed simultaneously and the calculatingtimes can be reduced.

Step 2 uses this technique in order to reduce the processing times andmake them compatible with real-time processing. It includes then, asindicated in FIG. 2, the following three sub-steps:

Step 2.1 “Δ”: pyramidal decomposition of the first intermediate imageinto first sub-sampled intermediate images Log(I_(INSSE)(p)).

Step 2.2 “FILTERING”: as above, filtering a first sub-sampledintermediate image using the low-frequency filter in order to obtain asecond sub-sampled intermediate image BF_(SSE).

Step 2.3 “∇”: pyramidal recomposition of the second sub-sampledintermediate image in order to obtain the second intermediate image withthe reference, as above, BF.

The other image processing steps remain unchanged.

In the context of video processing, it is advantageous to parameterizethe linear combination with the values of the minimum intensity(Min_(BF)) and the values of the maximum intensity (Max_(BF)) which areobtained in the immediately preceding image frame(s) in order to be ableto speed up the image processing.

The processing method according to the invention is particularlysuitable for the night vision systems including a low-light level imagecapturing device. Indeed, these systems provide generally noisy images,given the low level of lighting that is, therefore, necessary toprocess.

What is claimed is:
 1. A method of processing an initial digital imageI_(IN)(p) made up of pixels (p), said method comprising: calculating alogarithm of the initial image in order to obtain a first intermediateimage Log(I_(IN)(p)); filtering said first intermediate image using alow-frequency filter in order to obtain a second intermediate imageBF(p); determining a value of the minimum intensity (Min_(BF)) and avalue of the maximum intensity (Max_(BF)) in the low-frequency imageobtained from the filtering; calculating a third intermediate imageLog(I_(OUT)(p)) using a linear combination comprising:Log(I _(OUT)(p))=[Log(I _(IN)(p)−BF(p)]·[K1·BF(p)+K2]+K3·BF(p)+K4, K1,K2, K3 and K4 being constants dependent upon a difference between thevalue of the maximum intensity (Max_(BF)) and the value of the minimumintensity (Min_(BF)); and calculating a final image I_(OUT)(p) byapplying an exponential function to a third intermediate imageLog(I_(OUT)(p)).
 2. The method according to claim 1, wherein thelow-frequency filter is a Gaussian filter, the sigma of which is greaterthan
 30. 3. The method of claim 1, wherein the filtering furthercomprises: performing pyramidal decomposition of the first intermediateimage into first sub-sampled intermediate images Log(I_(INSSE)(p));filtering a first sub-sampled intermediate image using the low-frequencyfilter in order to obtain a second sub-sampled intermediate imageBF_(SSE)(p); and performing pyramidal recomposition of the secondsub-sampled intermediate image in order to obtain the secondintermediate image BF(p).
 4. The method of claim 1, wherein, in the casewhere the method is used to process a second digital image immediatelysucceeding a first digital image, both images belonging to a videosequence, and wherein the value of the minimum intensity (Min_(BF)) andthe value of the maximum intensity (Max_(BF)) of the second digitalimage are chosen to be equal to the value of the minimum intensity(Min_(BF)) and to the value of the maximum intensity (Max_(BF)) of thefirst digital image.
 5. A night vision system comprising: a low-lightlevel image capturing device; and a processor to process an initialdigital image I_(IN)(p) made up of pixels (p) captured from said deviceby: calculating a logarithm of the initial image in order to obtain afirst intermediate image Log(I_(IN)(p)); filtering said firstintermediate image using a low-frequency filter in order to obtain asecond intermediate image BF(p); determining a value of the minimumintensity (Min_(BF)) and a value of the maximum intensity (Max_(BF)) inthe low-frequency image obtained from the filtering; calculating a thirdintermediate image Log(I_(OUT)(p)) using a linear combinationcomprising:Log(I _(OUT)(p))=[Log(I _(In)(p)−BF(p)]·[K1·BF(p)+K2]+K3·BF(p)+K4, K1,K2, K3 and K4 being constants dependent upon a difference between thevalue of the maximum intensity (Max_(BF)) and the value of the minimumintensity (Min_(BF)); calculating a final image I_(OUT)(p) by applyingan exponential function to a third intermediate image Log(I_(OUT)(p)).